Evaluation of the AIMS System

This report describes the results of a set of experiments conducted with AIMS (Assisted Indexing at Mississippi State), a system that aids human document analysts in the assignment of indexes to physical chemistry journal articles. We defined four different experiments using four different combinations of portions of an article (such as the title and the abstract) to determine which portions of an article were most useful in the generation of indexes. Using a testbed of about twenty articles, we generated indexes for each article, then compared the AIMS-generated indexes with the indexes that had previously been assigned to the articles by the document analysts. The indexes generated by the document analysts are considered to be the “correct” indexes. We measured the success of AIMS in generating the correct indexes by making use of measurements from the area of information retrieval recall, precision, and overgeneration. In this report, we describe the various experiments and the success rates for AIMS in each of them.